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ABSTRACT 



This paper examines the impact of vouchers on student test 
scores in Dayton, Ohio, New York, New York, and Washington, DC, highlighting 
New York City parental assessment of private and public schools to 
investigate why vouchers seem to have differential effects depending on the 
students' ethnic background. Researchers collected baseline test scores and 
family data prior to the voucher lotteries, administered the lotteries, and 
collected follow-up information 1 and 2 years later. At pretest and posttest, 
students entering grades 1-8 took the Iowa Test of Basic Skills in reading 
and mathematics. Parents completed surveys regarding satisfaction with their 
children's schools, involvement in students' education, and demographics. 
Results varied systematically by ethnicity. There were no significant 
differences between the test scores of non-black students who moved to 
private schools and of the control-group students. Vouchers made a 
substantial, positive difference for black students but did not significantly 
impact Hispanic American students' test scores. All control group parents 
were satisfied with their children's education. Hispanic and black parents 
were very satisfied with private school placements, though they differed in 
perceptions . Private school impact on black students was not due to school 
size, class size, school disruptions, school communications, desegregation, 
dress rules, hallway monitoring, school resources, homework, parent 
involvement, or student involvement with the school. (Contains 10 tables and 
21 endnotes.) (SM) 
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Just ten years ago, the only data available about school vouchers came from an 

« 

experimental public-choice program conducted during the 1960s in Alum Rock, 
California.' But the early and mid-1990s brought new privately and publicly funded 
voucher programs to cities such as Milwaukee; Dayton, Ohio; Cleveland; Indianapolis; 
San Antonio; Washington, D.C., and New York City. With them has come a wealth of 
new research opportunities. 

Three were particularly advantageous. The privately funded voucher programs in 
Dayton, New York City, and the District of Columbia awarded vouchers by lottery, thus 
creating ideal conditions for a randomized field trial. Prior to conducting the lotteries, 
our evaluation team collected data on student test scores and family background 
characteristics. One and two years later, we retested the students. Since the abilities and 
family backgrounds of the test and control groups were, on average, similar before they 
entered the lottery, subsequent differences observed between the lottery winners and 
losers may be attributed to the effects of receiving a voucher. Because of their design, 
our evaluations of the Dayton, New York City, and Washington, D.C. voucher programs 
have yielded the best available information on students’ test-score outcomes and parental 
assessments of public and private schools. 

Elsewhere we have reported the impact of vouchers on student test-score 
performance in all three cities, finding positive effects of vouchers on African-American 
test scores but no effects on the test scores of students from other ethnic backgrounds.^ 

In this paper we summarize these findings, then examine parental responses in New York 
City in order to see whether we can explain why vouchers seem to have differential 
effects depending on the students’ ethnic background. 



er|c 



3 



Prior Research 



Our study is not the first to find that private-school effects are concentrated among 
African American students. The first indication comes from the High School and Beyond 
data collected by the Department of Education in 1980 and 1982. By surveying and testing 
a national sample of public and private schools in two waves, the Department of Education 
generated data on the determinants of gains in high school between a student’s sophomore 
and senior years. In a 1985 issue oi Sociology of Education, three particularly trenchant 
essays analyze and interpret these data."^ 

The authors of these essays noted serious disagreements about aggregate private- 
school impacts. Thomas Hoffer, Andrew Greeley and James Coleman found substantial, 
positive private-school effects on student test performance, while Douglas Wilms found 
trivial effects, if any. Christopher Jencks mediates the conflict, reaching judicious 
conclusions somewhere in the middle. 

Few observers, however, noticed in the Sociology of Education disputation the 
discussion of the effects of private schools on minority students. Hoffer, Greeley and 
Coleman found especially strong positive effects on low-income, minority students. 

Catholic schooling increased test scores by 4.4 answers, as compared to an impact of 1.6 
answers for students generally.^ Jencks shows that Wilms’s data, despite its exclusion of 
dropouts, also contain positive (though not statistically significant) effects of attending a 
Catholic school on African Americans’ reading scores. Taking all of the evidence from both 
studies into account, Jencks concludes that “the evidence that Catholic schools are especially 
helpful for initially disadvantaged students is quite suggestive, though not conclusive.”^ 
While overall impacts remain contested, those on minorities appear more robust. 
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Subsequent studies tend to reaffirm Hoffer, Greeley, and Coleman’s findings. In an 
analysis of the National Longitudinal Survey of Youth, Derek Neal finds that students who 
attend Catholic schools are more likely to graduate from high school and college. The 
effects, Neal notes, are the greatest among urban minorities. Catholic schools also have a 
significant, positive effect on black earnings potential, but not whites’.^ In separate studies, 
David Figlio and Joseph Stone and William Evans and Robert Schwab generate consistent 
findings for African Americans. After reviewing the literature concerning the effects of 
schools on minorities. University of Wisconsin Professor John Witte concludes that studies 
"indicate a substantial private school advantage in terms of completing high school and 
enrolling in college, both very important events in predicting future income and well-being. 
Moreover, ... the effects were most pronounced for students with achievement test scores in 
the bottom half of the distribution."^ 

Because they draw upon national datasets, all of these studies are of particular 
interest. One cannot rule out the possibility, however, that the observed positive effects 
were due to selection bias, a problem that arises when a population differentiates itself by 
freely choosing the treatment condition, in this case, attending a private school. This 
problem may be quite serious if those families with children in private schools looked very 
different from those with children remaining in public schools. Most of these studies 
adjusted for observable family background characteristics, such as mother’s education, 
family income, and other demographic factors. Yet one cannot be sure that the adjustments 
adequately account for an intangible factor — the willingness of a family to pay for their 
child's tuition, and all that this implies about the importance they place on education. Others 
performed two-stage regression models to reduce potential selection bias. But it is not easy 
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to find instrumental variables that are correlated with the type of school students attended 
but unrelated to the error term in the second-stage equation, as the disagreements between 
Neal and Figlio and Stone attest to. 

The best solution to the self-selection problem is the random assignment of 
students to test and control groups. Until recently, most evaluations of voucher programs 
have not utilized a random-assignment research design and therefore have not overcome 
the possible selection problems. Privately funded programs in Indianapolis, San Antonio, 
and Milwaukee admitted students on a first-come, first-served basis. And in the state- 
funded program in Cleveland, though scholarship winners were initially selected by 
means of a lottery, eventually all applicants were offered a scholarship, thereby 
precluding the conduct of a randomized experiment. The public Milwaukee program did 
award vouchers by a lottery, but data collection was incomplete.'*^ 

As a consequence, the findings presented here on New York, Dayton, and D.C. 
provide a unique opportunity to examine the effects of school vouchers on students from 
low-income families who live in central cities. In contrast to prior studies, the evaluation 
team conducted the lotteries. Follow-up test-score information was obtained from about 
one-half to two-thirds of the students who participated in the lottery, and baseline data 
provided information that allowed the analysts to adjust for non-response. 

The Programs 

In several key respects, the three voucher programs followed similar designs. All 
were privately funded; all were targeted at students from low-income families, most of 
whom lived in the inner-city; all provided only partial vouchers that the families were 
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expected to supplement; and all of the students in the evaluations of these three programs 
previously had been attending public schools. 

New York City. The School Choice Scholarships Foundation (SCSF) in New York City 
offered 1,300 scholarships worth up to $1,400 annually toward tuition at a private school 
for at least three years. To qualify for a scholarship, children had to be entering grades 1 
through 4, live in New York City, attend a public school at the time of application, and 
come from families with incomes low enough to qualify for the U.S. government’s free 
school-lunch program. More than 20,000 students applied between February and late 
April 1997. By the end of the scholarship program’s second year, 64 percent of the 
lottery-winning students were attending a private school. 

Dayton, Ohio. In the spring of 1998, Parents Advancing Choice in Education (PACE) 
offered low-income students in grades K-12 the opportunity to win a scholarship to 
attend private school. For the 1998-99 school year, PACE offered scholarships to 515 
students who were in public schools and to 250 who were already enrolled in private 
schools in the Dayton metropolitan area. During the program’s first year, the PACE 
scholarships covered 50 percent of tuition at a private school, up to $1,200. Support was 
guaranteed for at least four years, with a possibility of continuing through high school, 
provided funds remained available. Of those students offered scholarships, ** percent 
enrolled in a private school during the second year of the program. 

Washington, D.C. The Washington Scholarship Fund (WSF), originally established in 
1993, is the oldest of the three programs. By the fall of 1997, the WSF was serving 
approximately 460 children at 72 private schools. After receiving a large infusion of new 
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funds from two philanthropists, however, the WSF announced a major expansion in 
October 1997. 

To qualify, applicants had to reside in Washington, D.C., and he entering grades 
K-8 in the fall of 1998. Families with incomes at or helow the poverty line received 
vouchers that equaled 60 percent of tuition, or $1,700, whichever was less. Families with 
income above the poverty line received smaller scholarships. Families with incomes 
higher than two-and-a-half times the poverty line were ineligible. The WSF claims that 
it will maintain tuition support for at least three years and, if funds remain available, 
until students complete high school. In April 1998, the WSF awarded more than 1,000 
scholarships by lottery, with the majority going to students previously attending a public 
school. Of those students offered scholarships, 35 percent were still using them to 
attend a private school in the second year of the program. 

Evaluation Procedures 

The evaluation procedures used in all three studies conformed to those in 
randomized field trials. Our evaluation team collected baseline test score and family 
background information prior to the lottery, administered the lottery, and collected 
follow-up information one and two years later. 

Students took the Iowa Test of Basic Skills (ITBS) in reading and mathematics. 
Students who were entering grades 1-4 in New York City and grades 2-8 in Dayton 
(and other parts of Montgomery County, Ohio) and Washington, D.C., were included in 
the evaluations. Parents filled out surveys on their satisfaction with their children’s 
schools, their involvement in their children’s education, and their demographic 
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characteristics. Students in grades 4 and higher completed similar surveys. In all three 
cities, the follow-up procedures replicated the pre-lottery procedures: students again 
took the ITBS in reading and math; parents and older students filled out surveys about 
their backgrounds and educational experiences. 

More than 5,000 students participated in pre-lottery testing in New York City. 

Of the families did not win the lottery, approximately 1,000 were selected at random to 
comprise a control group of approximately 960 families. All of these students were 
attending public schools at the time. In Dayton, 1,440 students were tested pre-lottery, 
803 of whom were attending public schools at the time. In Washington, D.C., 2,023 
students were tested pre-lottery, of whom 1,582 were attending a public school. 

Separate lotteries were held in Dayton and D.C. for students who were enrolled in public 
and private schools; because only public-school children were eligible to apply for a 
scholarship in New York, there was no need to hold separate public and private lotteries 
there. In all three cities, only those students who were in public schools at the time of the 
lottery are included in this study. 

In New York City, 42 percent of the students participating in the second year of 
the evaluation were Afiican-Americans; in Dayton, 47 percent; and in D.C., 94 percent. 
Hispanic students accounted for 51 percent of the New York City population, 2 percent 
of Dayton’s, and 4 percent of D.C. ’s. Whites accounted for 5 percent of New York 
City’s evaluation group, versus 24 percent in Dayton, and 1 percent in D.C. The 
remaining students came from a variety of other ethnic backgrounds. 

In New York City, 80 percent of the students included in the evaluation attended 
the first-year testing sessions; 66 percent attended the second-year sessions. In D.C., the 
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response rate after one year was 63 percent; after two years, it was 50 percent. In 
Dayton, 57 percent of families attended follow-up sessions after one year, and 49 
percent after two years. 

We are reasonably confident that these modest response rates do not undermine 
the integrity of our findings. First, with the exception of the second year in New York, 
response rates were similar for treatment and control groups after one and two years in 
all three cities. Second, comparisons of baseline test scores and background 
characteristics revealed only minor differences between the second-year respondents and 
nonrespondents in all three cities. Finally, to account for the minor differences between 
respondents and nonrespondents that we did observe, the test scores of children who, 
based on their demographic characteristics, were more likely to attend follow-up 
sessions were weighted less heavily, while the test scores of children who were less 
likely to attend follow-up sessions, but nevertheless did, were weighted more heavily. 
Given the slight differences between respondents and nonrespondents, however, the 
weights had little effect on the results. 

The randomized lottery ensured that lottery winners as a group were not 
significantly different from the control group (those who did not win a scholarship). In 
all three cities, the demographic characteristics and pre-lottery test scores of scholarship 
winners and losers (the treatment and control groups, respectively) were identical to one 
another. Only in Dayton were there minor differences in the pre-lottery test scores: those 
offered a voucher scored 6.5 percentile points lower in math and 3.1 points lower in 
reading than those not offered a scholarship, a statistically significant difference. 




10 



10 



To measure the impact on children’s test scores of switching to a private school, 
we estimate a statistical model that takes into account whether a child attended a public 
or private school, as well as baseline reading and math test scores. Baseline test scores 
were included to adjust for the minor baseline differences between the treatment and 
control groups on the achievement tests and to increase the precision of the estimated 
impact. 

The lottery generated two groups: those who were offered a voucher and those 
who were not. We’re not interested, however, in the effect of being offered a voucher. 
Rather, we’re interested in the effect of using a voucher to attend a private school. A 
simple comparison between public and private school students, however, is inappropriate 
because certain students may be more likely to take advantage of a voucher. Their 
parents might place greater value on education and be more willing to supplement the 
voucher, or they may live in a neighborhood with a broader selection of private schools. 

If these children differ from students who won a voucher but failed to use it in ways that 
are related to student achievement, it could bias our findings. To solve this problem, we 
used as an instrumental variable whether or not individual was offered a voucher to 
predict the probability that she attended a private school; with these predicted values, we 
then estimated the actual impact of switching from a public to a private school. This two- 
stage regression technique was first used in medical research and is now commonplace in 
econometric studies. 



Test Score Results 

Our findings varied systematically according to the student’s ethnicity. In all 
three cities, there were no significant differences between the test-score performance of 
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non- African-American students who switched from a public to a private school and the 
performance of students in the control group — either after one or two years (Table 1)." 
For African-American students, however, vouchers made a substantial difference. In the 
three cities combined, African American students who switched from public to private 
schools scored, after one year, 3.3 percentile points higher on the combined math and 
reading tests (expressed as National Percentile Ranking (NPR) points, which run from 0 
to 100 with a national median of 50). After two years, African-American students who 
used a voucher to enroll in a private school scored 6.3 percentile points higher than their 
public school peers. 

Table 1 also shows that the largest voucher impacts for African-American 
students were observed in the Washington, D.C. program. African American students 
who attended D.C. private schools for two years scored 9.0 percentile points higher than 
students in the control group. The smallest differences after two years were observed in 
New York City, African- American students attending private schools scored 4.3 
percentile points higher than the control group. In Dayton, the difference between test 
and control groups was 6.5 percentile points. 

The average impact of vouchers on the test scores of African-Americans was 
moderately large. After one year, black students who switched to private schools scored 
.17 standard deviations higher than the students in the control group. After two years, the 
difference grew to .33 standard deviations, roughly one-third of the test-score gap 
between blacks and whites. 

The magnitude of the effects can be further assessed by comparing them to the 
effects observed in an evaluation of a class-size reduction intervention conducted in 
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Tennessee, the only other major education reform to be subjected to evaluation by means 
of a randomized field trial. The effects on African Americans of attendance at a private 
school shown here are larger than the estimated effect of a 7-student reduction in class 
size. According to a recent reanalysis of data fi'om Termessee, the class-size reduction 
effect for Afiican Americans after two years was, on average, 4.9 percentile points, 
somewhat less than the 6.7 percentile effect of switching to a private school.’^ 

It is also of interest to compare the size of the effects of the voucher intervention 
with the size of the effects reported in the RAND study entitled Improving School 
Achievement released in August 2000. Identif)dng the most successful states, Texas 
and North Carolina, which have introduced rigorous accountability systems that involve 
state-wide testing, the study finds what it says are “remarkable” one-year gains [in math 
scores] in these states of “as much as 0.06 to 0.07 standard deviation[s] per year” — or 
0.12 to 0.14 over two years. The two-year effects of the school voucher intervention on 
black students observed here are over twice as large. 

Controlling for Demographics 

Most research on the impact of private schools attempts to control for differences 
in family income and other background characteristics among students attending public 
and private schools. When a lottery is used to separate research subjects into 
experimental and control groups, however, such statistical adjustments are generally 
urmecessary, given that the two groups being compared are virtually identical to one 
another. 

Nonetheless, a number of analysts have objected to the apparent absence of 
controls for family background characteristics. Bmce Fuller and his colleagues at the 
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University of California, Berkeley, for instance, argued that “the experimental group may 
have been biased as some of the most disadvantaged voucher winners did not switch to a 
private school, and therefore were excluded from the group (possibly boosting mean 
achievement levels artificially).” An interest group. People for the American Way, lodged 
a similar complaint: “The . . . study’s key finding improperly compares two dramatically 
different groups and may well reflect private-school screening-out of the most at-risk 
students.” 

In the three cities, roughly half the students took the voucher that was offered to 
them (the takers) and about half did not (the decliners). However, we did not drop the 
decliners from the analysis. All voucher applicants were invited to follow-up testing 
sessions, and each of the families who participated is included in the analysis. To 
estimate the impact of switching from a public to a private school, we did not simply 
compare takers with the control group (the decliners and those who didn’t win a 
voucher), as Fuller and his colleagues have contended. Instead, as previously noted, we 
used the fact that the vouchers were awarded randomly to generate an instrumental 
variable that generates an unbiased estimate of the effect of switching to a private school. 

Even if one were to control for family background characteristics, the results are 
unlikely to vary significantly. The use of a randomized lottery ensures that the 
background characteristics of lottery winners and losers will differ significantly. To show 
this, we recalculated the impact of attending a private school on test scores, this time 
including explicit controls for the mother’s educational level, her employment status, 
family size, and whether or not the family received welfare. And, as expected, the 
difference in the combined reading and math test scores of African Americans after two 
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years in all three cities remain exactly the same — 6.3 NPR points, a statistically 
significant impact. 

The Sore Loser Hypothesis 

Since releasing our study, some critics have argued that we have falsely attributed 

the observed gains in the treatment group to the positive impact of a voucher. Is it not 

possible that members of the control group are sufficiently frustrated by the experience of 

applying for a voucher, and then being refused one, that they no longer remain engaged 

in their child's education? New York Times columnist Richard Rothstein, for example, 

iterates a hypothesis first made by Stanford University Professor Martin Camoy. 

Parents know if their children got vouchers and this knowledge can affect results. 
For example, volunteers for vouchers, already more dissatisfied with public 
schools than others, may have their hopes raised, then dashed when they were not 
selected for a voucher. Sorely disappointed, they may then demand less of their 
children in public school.'"' 

We know that parents with children in private schools are much more satisfied with their 
child's school that public-school parents. Is it possible that the test score results that we 
have observed are due not so much to higher quality schooling in the private sector as to 
a deterioration in parental involvement in the control group? 

To explore this hypothesis, we examined the control group’s satisfaction levels at 
baseline, after one year, and after two years. On each of these occasions, parents were 
asked: "How satisfied are you with the following aspects of your child's current school?" 
Items included teaching, school safety, parental involvement, class size, school facility, 
student respect for teachers, communication regarding student progress, freedom to 
observe religious traditions, and the school's location.'^ Parents then were given four 
options, "very satisfied,” “satisfied,” “dissatisfied,” and “very dissatisfied.” The results 
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presented here come from New York City; in future papers we shall report findings for 
Dayton and D.C. 

In all cases, the levels of satisfaction expressed by members of the control group 
increased one year after having been denied a scholarship (table 2). At the end of two 
years, satisfaction levels did deteriorate somewhat. On five of the nine items, members 
of the control group reported being slightly less satisfaction about their public schools 
than they were at baseline. But even after two years the control-group parents remained 
as satisfied with their public schools as they had been prior to the lottery. 

Parental responses to questions about their relationships with their children cast 
further doubt on the hypothesis that the fhistration associated with losing the voucher 
lottery lead control-group parents to care less about the education of their children. 
Parents were asked how often they helped their child with homework, talked with their 
child about school, attended school activities, and worked on school projects. In every 
case, the answers given by parents with children in the public-school control group after 
both one and two years remained roughly constant, and closely resembled the responses 
of parents in the treatment group (Table 3).'^ 

These data lend little support for the claim that control-group parents were sore 
losers. Given that parents knew they had only about a one in twenty chance of winning 
the lottery, their initial expectations were probably not unduly high. It is hard to imagine, 
then, that whatever disappointment parents felt when they lost the lottery led to their 
children’s systematic underachievement when tested one and two years later. 




16 



16 



Hawthorne Effects 

As a corollary to the sore loser hypothesis, Camoy suggested that our findings 
might represent Hawthorne effects. If so, then the observed gains for African Americans 
may have little to do with vouchers per se, but rather the surge of enthusiasm associated 
with wirming a lottery. Upon learning that their children could now attend a private 
school, the interest and involvement of treatment-group parents in their children’s 
education may have been reinvigorated. A year later, such enthusiasm might wane and 
children could lose the family support they need to do well in school. 

To ascertain whether Hawthorne effects explain the pattern of results, we revisited 
our measures of parental satisfaction, again focusing on New York City. We constructed 
an overall index of satisfaction based upon parental responses to all of the parental 
satisfaction questions, each of which is scored from 1 to 4 depending on the level of 
satisfaction. The index was then rescaled to have a standard deviation of 1.0. By scaling 
the measure in this way, one can easily ascertain the “effect sizes” of attending a private 
school as compared to remaining in public school. 

When comparing the impact of attending a private school on parental satisfaction 
in years one and two, we find some support for the Hawthorne hypothesis. The effect 
size of attending a private school on the parental satisfaction of African Americans in 
year one was fully 1.2 standard deviations (see table 4). The effect size attenuates 
somewhat in year two, dropping to 1.0 standard deviation. Similarly, the size of the 
effect of attending a private school on the overall grade African American parents gave a 
school declines from 1.14 standard deviations in year one to 0.86 in year two. What is 
more, the impact on African American test scores in New York City decreases slightly 
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from year one to year two, suggesting that Hawthorne effects may he responsible for 

17 

some of the initial test-score gains. 

Other facts, however, cast douht on Camoy’s intuition. First, the impact of 
attending a private school on parental satisfaction was quite large in both years one and 
two. Even after two years, the effect size for parental satisfaction hovers around a full 
standard deviation. The 0.2 diminution of satisfaction among African American parents 
that occurred between year one and year two was rather modest when compared to the 
striking differences in satisfaction with private and public schools that remained after two 
years. 

Second, the impacts on satisfaction rates of Latino and African American parents 
were comparable in year one, and by year two the impact for Latinos was in fact slightly 
higher. If all it takes to elevate test scores is to enhance parental satisfaction with a 
school, then why haven’t Latinos posted significant test score gains? 

Third, while the trajectory of voucher impacts on test scores in New York City 
between years one and two declines slightly, in Dayton and D.C. it rises noticeably. 

When considered together, the overall test score gains observed in all three evaluations — 
New York, Washington, D. C., and Dayton, Ohio — climb from approximately 3 points in 
year one to 6 points in year two, a pattern that does not square with the Hawthorne effect 
hypothesis. 

Finally, while they may inform parental satisfaction rates, Hawthorne effects are 
less likely to drive student achievement. Students participating in voucher experiments 
are being asked to change schools, form new friendships, adjust to new rules and 
expectations, and acquire new study habits. In Washington D.C., for instance, older 
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students who transferred to private schools indicated intense resentment with these 
changes, which in turn was reflected in their first-year test scores. It is possible, then, 
that rather than reflecting Hawthorne effects, observed impacts will only increase as 
students have a greater amount of time to adjust to their new schools, and the educational 
expectations laid upon them. 

Thus far, the preponderance of the evidence suggests that test score gains 
experienced by African Americans are probably not due to Hawthorne effects. In the 
future, however, we should be in a stronger position to evaluate the merits of Camoy’s 
claim. If the gains observed thus far trace back to Hawthorne effects, then the impacts 
for African Americans observed thus far should subsequently attenuate. If the effects are 
real, though, the impact of attending a private school on test scores should increase. Data 
for the third- year evaluation have been collected and will be reported during 2001. 

Explaining Ethnic Differences in Voucher Impacts 

Neither the absence of background controls, nor the disappointment of losing a 
voucher, nor Hawthorne effects appear to explain away the observed gains for African 
Americans, at least after two years. A basic puzzle, therefore, arises. Why should 
vouchers have a positive impact on the test scores of Afiican American students, but not 
anybody else? This finding is particularly curious in New York, where Afiican American 
students posted positive and significant test score gains, but Latinos did not. As poor, 
minority residents of inner cities, both groups presumably face a common set of 
educational obstacles. One would think, then, that an intervention that successfully 
improves the test scores of one group would have a similar impact on the other. 
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We are best equipped to explore this question in New York City, where a fairly 
large number of both African America and Latino students participated in the evaluation. 
In Dayton, and especially in D.C., we simply do not have enough non- African American 
students to support a sustained analysis of the causal factors that affect the test score 
performances of different ethnic groups. 

The remainder of this paper draws upon parental survey data to assess a broad 
array of possible explanations for the observed differential race effects: language, school 
disruptions, class size, school size, parental communications, and a “kitchen sink” models 
that simultaneously control for a multiplicity of factors. We first test the impact of 
vouchers for Afiican Americans and Latinos on each of these aspects of a student’s 
education; using these results, we then select a subset of factors to include in the original 
test score models to see whether or not they reduce the positive effect of vouchers 
experienced by Afiican Americans. Unfortunately, they do not, neither singularly nor 
additively. In the end, we are better able to rule out possible explanations than draw 
positive conclusions. 

Language Needs 

The fact that Afiican Americans appear to benefit from vouchers, but Latinos do 
not, may have nothing to do with race per se, and everything to do with language. Private 
schools may be poorly equipped to deal with students who do not speak English as their 
primary language; public schools, meanwhile, often have well-established ESL programs 
and specially trained personnel to deal with the particular cultural and linguistic needs of 
minority populations. It is possible, then, that the gains associated with a private 




20 



20 



education may be transferred only to those students who can function in all-English 
classrooms. 

To test this hypothesis, we compared the impact of switching to a private school 
on the test scores of Latino students whose primary language (according to their parents) 
was English with those for whom English was a secondary language. As can be seen in 
Table 5, the results, if anything, run directly contrary to expectation. Non-English 
speaking Latinos post slightly positive impacts, while Latinos for whom English is the 
primary language post slightly negative effects. Neither the positive nor negative 
impacts, nor the slightly larger differences in impacts, are statistically significant. 

These findings do not provide much of a basis on which to judge the ways in 
which public and private schools deal with students with language needs. They do rule 
out language, however, as an explanation for why African Americans appear to benefit 
from vouchers, while Latinos do not. The next section, therefore, shifts focus and 
examines the characteristics of public and private schools that African Americans and 
Latinos attended. 

Parental Perceptions of Public and Private Schools 

Parents accompanied children to the follow-up testing sessions. Because the 
testing period took over an hour, parents had time to complete fairly lengthy 
questionnaires about the schools their children were attending. In previous papers, we 
reported the results from these surveys for all parents regardless of their ethnic 
background.'^ These results provide information concerning the impact of switching to a 
private school on parental perceptions of numerous aspects of school life. Generally 
speaking, in New York, Dayton, and D.C., we found that: 
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• Private schools have stricter dress codes. 

• Hallways in public schools are more closely monitored — students are more 
likely to need passes to leave the classroom and visitors are more likely to 
have to get permission slips. 

• School disruptions — fighting, cheating, property disruption, student 
misbehavior, truancy, tardiness, and so forth — are more extensive in public 
schools. 

• Suspension rates are similar in the two sectors. 

• Public schools have more physical resources — cafeteria, nurse’s office, 
gymnasium arid so forth. 

• Public schools have a greater variety of academic programs — special 
education, advanced education, bilingual education, and so forth. 

• Private schools communicate more with parents by means of teacher-parent 
conferences, parental participation in school, and so forth. 

• Students in private school do more homework. 

• Private schools have smaller classes 

• Private schools have fewer students. 

• Although results differ from city to city, on the whole the degree of 
segregation is similar in the two sectors. 

• Parents in both sectors are equally involved in their child’s education. 

• Parents in both sectors volunteer and participate equally in their child’s 
school. 

It is possible that the observed impacts of vouchers on some of these school 
characteristics vary for different ethnic groups, and therefore may represent likely 
candidates for why African Americans appear to benefit from choice, while Latinos do 
not. To explore this possibility, we estimated the impact of attending a private school in 
New York City separately affected African Americans and Latinos. Some aspects of 
school life — class size, school size, amount of time spent on homework, degree of ethnic 
segregation, and suspension rates — could be easily measured by using responses parents 
gave to a single question. When possible, though, we constructed indices from multiple 
survey questions that measured the same school characteristic. Questions used to 
generate each index are reported in the Appendix. 

We estimated the impact of switching to a private school on each aspect of school 
life in the same way that we estimated the impacts on test scores, except that now we do 
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not control for baseline test scores. The results of this investigation are reported in 
Tables 6; for parental involvement items, see table 3. 

For African Americans, the impacts of switching to a private school on parental 
perceptions of most aspects of school life were fairly stable from the first to the second 
year. In both years, African American parents in private schools reported significantly 
fewer school disruptions (fighting, cheating, property destruction, and so forth) than 
parents in public schools. They also reported significantly more demanding dress codes, 
significantly less hallway monitoring, fewer school resources, greater parental 
communication by the school, more homework, smaller schools, and slightly less 
involvement in the school (though this difference is statistically significant only in year 
two). Afiican American parents in private schools, as compared to those in the public- 
school control group, also reported no differences in suspension rates, in their 
involvement with their child’s education, in the likelihood that their child attended a 
segregated school, and the number of specific programs (bilingual education special 
education, advanced education, and so forth) at their child’s school. In only one respect 
did the impacts reported by Afiican American parents differ in years one and two. In 
year one, Afiican Americans in private schools reported that their child was in a 
significantly smaller class, but not in year two. 

In some respects, the results for Latino parents reveal similar trends. Latino 
parents in private schools report stricter dress codes, more communication with their 
schools, more homework, and smaller schools. They also report no difference between 
the two sectors in suspension rates, the range and variety of school programs, and their 
own involvement with their child’s education. 
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In several other respects, however, school vouchers appeared to have very 
different effects on the educations of Latino students. Latino parents who attended 
private schools, for instance, do not report a reduction in the number of school 
disruptions. Nor do they report smaller classes or fewer school resources than the control 
group in either year. Only in one year do Latino parents in private schools report less 
hallway monitoring. And in one year they report less segregation in the private sector. 

The last column of tables 3 and 6 identifies whether or not the observed impacts 
for Afiican Americans and Latinos are statistically significantly different from one 
another. Those items with stars represent plausible components of an explanation for 
why Afiican Americans appear to benefit from vouchers, but Latinos do not. Four 
aspects of school life stand out — school size, parental communications, class size and 
school disruptions. In both years, vouchers had a smaller impact on the size of the 
private schools and classrooms attended by Latino students than they did on those 
attended by African Americans. Also, while vouchers had a large and positive impact on 
the communication levels of Afiican Americans, they had relatively small impact on 
those of Latinos. And perhaps most strikingly, the magnitude of the impacts of attending 
private school on school disruptions varied dramatically for Latinos and African 
Americans. Given the sizes and signs of these differences in impacts, these four factors 
become prime suspects for explaining the differential race effects of vouchers on test 
scores. 

Other differences are evident in one of the two years. In year two the impact of 
attending a private school on hallway monitoring was significantly smaller for Latinos 
than it was for Afiican Americans. The year-two impact on school resources, meanwhile. 
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was larger. Given the sign of these differences, however, they probably do not explain 
why African American students voucher students log the only test score gains. Could it 
be that African American students in private schools benefit from the fact that their 
hallways are less closely monitored? Do black students in private schools benefit 
because they have fewer school resources — cafeteria, gymnasium, nurse’s office, and the 
like? Probably not. 

Note that from these impacts we cannot infer whether African Americans are 
coming from a particularly poor lot of public schools, or are gaining access to a 
particularly effective group of private schools. All that we know is that along some 
dimensions, the impact of the switch for African Americans was greater than that for 
Latinos. With our short list of four factors that may explain the observed differential race 
effects, however, we can now proceed to the second step in this analysis — showing the 
relationship between these factors and student achievement. 

Do School Size, Class Size, School Disruptions, and Communication Levels Explain 
Why Vouchers in New York City Lift African Americans’ Test Scores, but 
Latinos’? 

To check, we regressed class size, school disruptions, school size, and school 
communications on student test scores after both the first and second year. Also included 
in the equation was an indicator variable for whether or not the student had been offered a 
voucher, as well as baseline test scores and lottery indicators. If these four school 
characteristics explain the differential impact of a voucher offer on the two ethnic groups, 
the impact of the voucher on African Americans should diminish or entirely disappear 
once they are added to the model. 
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The results for African Americans are reported in Table 7. The first column 
reports the impact on African American test scores of being offered a voucher in New 
York City after one year: 4.6 percentile points. In column three, we report the same 
effect after two years, 3.3 percentile points.' 

In columns two and four, we report the effects of a voucher offer in years one and 
two, respectively, controlling for parental reports on school disruptions, school 
communications, class size, and school size. If these factors explain the impact of a 
voucher on African American test scores, then the effect of a voucher should diminish or 
disappear. Unfortunately, this does not happen. Instead, the size of the impact remains 
essentially constant. These four factors, neither separately nor combined, do not explain 
why African Americans perform better on tests when given an opportunity to attend a 
private school. Parenthetically, only one of the four items in either year — the school 
disruption index in year two — has a significant and direct impact on African Americans’ 
test scores. The others do not appear to have any causal impact at all. 

Table 8 provides a similar set of equations for Latino students. The offer of a 
voucher has no impact on student performance either before or after these items are 
included in the equation. In addition, school disruptions appear to have a negative direct 
impact on student performance, especially in year one. Class size also has an effect, but 
its sign is perverse: Latinos do better in larger classes. This correlation could be caused. 



' (Please note that these equations estimate the effect of a voucher offer, not that of actually switching to a 
private school, as reported in Table 1. The latter estimates are larger because not everyone offered a 
voucher makes use of one, and some families who are not offered a voucher attend a private school 
anyway. For two reasons, this section focuses on the effects of a voucher offer. First, because of the 
random process by which vouchers were awarded, the estimate of a voucher offer can be recovered with a 
simple ordinary least squares equation. And second, there is a linear relationship between these two 
estimates: the impact of switching from a public to a private school uses as a benchmark the impact of a 
voucher offer, and adjusts the estimate upwards or downwards depending upon the percentage of students 
in the treatment and control groups that attend private schools.) 
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however, by the assignment of Latino students with language or learning difficulties to 
smaller sized classes. 

The technically more sophisticated reader may wish to see these same results 
presented in one equation that includes interaction terms. This information is provided in 
table 9. The regression includes the overall impact of a voucher offer on both groups of 
students (which in this equation specifies the impact on Latinos), the interaction between 
a voucher offer and whether or not the student is an Afiican American (the sum of the 
estimated coefficients on this interaction term and the main treatment variable is the 
impact on Afiican Americans), an indicator variable for African Americans, and separate 
terms for school disruptions, school communications with parents, school size, and class 
size. Also included in the equation are controls for baseline test scores, other variables 
necessitated by the particular features of the lottery administered in New York City, and 
all of their respective interaction terms. 

The estimated effects in this saturated model replicate those recovered when 
running separate models for Afiican Americans and Latinos. As before, these factors do 
not explain the impact of vouchers on African American test scores; the direct effects of 
these four variables remain the same as well. Interestingly, though, the indicator variable 
for Afiican American is negative and highly significant, suggesting that African 
Americans, on average, scored lower than their Latino peers. 

Do All Measured Factors, Taken in Combination, Explain the Impact of Vouchers 
on African American Test Scores? 

Perhaps the factors that impact Afiican American test scores are not the same 
ones that distinguish the impact of vouchers on the perceptions of Afiican American and 




27 



27 



Latino parents. Perhaps it is some other factor or all factors combined that account for 
the differential race effects that we observe. To see whether or not this is the case, we 
conducted a kitchen sink analysis, one that included every survey item in a single model, 
along with all relevant interaction terms. Such a model is not a very good way of 
estimating the impact of any particular aspect of school life on student test score 
performance. Because few of the indices measure distinct school characteristics, the 
estimated impact of each is being partially estimated by others. However, this approach 
allows us to ascertain whether measurable aspects of school life help explain the private 
school advantage for African Americans. 

Table 10 answers the question in the negative. Even when we include all items — 
not only school disruptions, school communications, school size, and class size but also 
suspensions, dress rules, hallway monitoring, school resources, homework, segregation, 
parental involvement with child, and parents involvement with the school — in the 
equation, the impact of the voucher offer on African American test scores remains intact. 



Discussion 

If African Americans learn more in New York City private schools than they do 
in public schools, and if the private-school impact is not due to school size, class size, 
school disruptions, school communications, desegregation, dress rules, hallway 
monitoring, school resources, homework, level of parental involvement with child’s 
education, and their involvement with the school, then what does explain the difference? 

Parental perceptions are not always as precise as we might like, and so we need 
not prematurely rule out these school characteristics as possible explanations for the 
differential race effects that vouchers seem to generate. Still, though, we remain 
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impressed how similar are the patterns of parental response from one year to the next and 
one city to another. If parents were responding to questions more or less randomly, then 
we should observe different patterns that vary across city and over time. Instead, the 
pattern of parental responses is remarkably stable. 

It is possible, though, that the voucher impacts derive not from these item 
considered separately or additively, but through some complex interaction among some 
or all of the variables that our models have not estimated. Perhaps it is the interaction 
between school disruptions and school size that counts? Or the interaction between 
parental-school communications and class size? Indeed, different aspects of school life 
may come together in different ways for African Americans and Latinos, generating very 
different test scores outcomes for the two groups. In the future, we plan to estimate 
additional models that account for the interaction between different school factors, rather 
than just the main effects of each taken one at a time. 

It is also possible that private-school impacts are due to instructional factors that 
none of the items in our parental survey adequately measure. Perhaps the disparities 
between the quality of teachers for African American students in public and private 
schools are much wider than those for Latino students. Perhaps African American 
students are particularly and uniquely receptive to teaching techniques that are more 
prevalent in private schools. Recent research has shown that teacher effectiveness can 
have a large impact on student test-score performance. Our models, however, do not 
include any measures of curriculum, teaching techniques, the expectations that teachers 
place on their students, or teacher quality. Such factors might be the key to 
understanding why African Americans benefit from choice, but Latinos do not. 
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Finally, the effects may have nothing to do with the characteristics of public and 
private schools that African Americans and Latinos attend. They may instead derive 
from the quality of the peer groups at these schools. Richard Rothstein thinks positive 
effects arise when voucher recipients “are surrounded by pupils with higher academic 
expectations.”^' If African Americans attend private schools with a particularly elite 
group of classmates, while the peer groups of Latinos who switch from public to private 
school change very little, then peer effects may lay at the heart of the story we are trying 
to uncover. 

We still do not know what it is about private schools that make them successful, 
at least for African Americans. And without an answer, it remains unclear how, or even 
whether, public schools can introduce appropriate reforms that clearly and identifiably 
benefit African American students. Future pilot studies that contain a larger number of 
subjects and collect a broader array of information may unearth some of the reasons why 
at least some students appear to benefit from choice. A good place to begin this research, 
it seems to us, would be in the District of Columbia, where voucher impacts were large 
and the population to be served is largely African American. 
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TABLE 1: Impact of Switching to a Private School on Test Score Performance 





Year 1 
(Percentiles) 


(N) 


Year 2 
(Percentiles) 


(N) 


African Americans 










New York City 


5.8** 


623 


4.3** 


497 


Dayton, Ohio 


3.3 


296 


6.5* 


273 


Washington, D.C. 


-0.9 


891 


^ Q*9(C9|C 


700 


Average Impact 


3.3 




6.3** 




All Other Ethnic Groups 










New York City 


-1.4 


704 


-0.9 


612 


Dayton, Ohio 


1.0 


108 


-0.2 


96 


Washington, D.C. 


7.4 


39 


0.1 


44 


Average Impact 


0.2 




-0.8 





* significant at .10 level, 2-tailed test; ** .05 level; *** .01 level. Weighted two-stage least squares 
regressions performed; treatment status used as instrument. Impacts expressed in terms of national 
percentile rankings. In New York City, 2.8 percent of the African American control group in the 
year 2 models attended a private school for one of two years; in Dayton, 2.0 percent of the African 
American control group in the year 2 models attended a private school in the second but not the 
first year; and in D.C., 3.7 percent of the African American control group in the year 2 models 
attended a private school in the second year but not the first year. In New York City, all non- 
African Americans in this study consist of Latinos; in Dayton, the vast majority consists of 
Caucasians. 
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Table 2. — Percentage of Control Group 'Very Satisfied' with their Public School at 

Baseline and after One and Two Years 



How satisfied are you with the following aspects of your child's current school? 



% very satisfied with: 


1997: 

Baseline 


1998: 
Year One 


1999: 
Year Two 


Teaching: 


14 


23 


10 


School Safety: 


13 


21 


9 


Parental Involvement: 


11 


19 


12 


Class Size: 


7 


12 


7 


School Facility: 


9 


14 


5 


Student Respect for Teachers: 


18 


21 


11 


Communication Regarding 
Student Progress 


18 


23 . 


19 


Freedom to Observe 
Religious Traditions 


8 


9 


5 


Location 


25 


34 


28 



Baseline satisfaction scores are for all families not offered a scholarship from which control group is drawn. 
Year-one and year-two satisfaction scores are weighted so that generalizations can be made to the baseline 
applicant pool. 
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TABLE 3: Impacts of Switching to a Private School on Measures of Parental Involvement 
of African Americans and Latinos in New York City 





African Americans 
(Percentiles) 


(N) 


Latinos 

(Percentiles) 


(N) 


Diff. In 
Impacts 


Parental Involvement with 
Child’s Education 
Year 1 Impact 


-0.04 


[0.11] 


532 


-0.11 


[0.15] 


568 




Year 2 Impact 


-0.06 


[0.13] 


470 


-0.15 


[0.16] 


529 


— 


Parental Involvement with 
Child’s School 
Year 1 Impact 


-0.14 


[0.14] 


519 


0.04 


[0.14] 


553 




Year 2 Impact 


-0.23* 


[0.14] 


470 


-0.08 


[0.15] 


529 


— 



* significant at .10 level, 2-tailed test; ** .05 level; *** .01 level. The last column denotes whether the difference 
in the estimated impacts for African Americans and Latinos is statistically significant. Standard errors reported in 
parentheses. Weighted two-stage least squares regressions performed; treatment status used as instrument. All 
models control for lottery indicators. Impacts expressed in terms of effect sizes. 
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TABLE 4: Impacts of Switching to a Private School on Measures of Satisfaction for 
African Americans and Latinos in New York City 





African Americans 
(Percentile^ 


(N) 


Latinos 

(Percentiles) 


(N) 


Diff. In 
Impacts 


Satisfaction Index 
















Year 1 Impact 


1.23*** 


[0.12] 


533 


0.98*** 


[0.13] 


567 


** 


Year 2 Impact 


1.00*** 


[0.12] 


466 




[0.14] 


529 


— 


Overall Grade 
















Year 1 Impact 




[0.12] 


533 


0.75*** 


[0.13] 


567 




Year 2 Impact 


0.86*** 


[0.13] 


469 


1 01*** 


[0.14] 


530 





* significant at .10 level, 2-tailed test; ** .05 level; *** .01 level. The last column denotes whether the difference 
in the estimated impacts for African Americans and Latinos is statistically significant. Standard errors reported in 
parentheses. Weighted two-stage least squares regressions performed; treatment status used as instrument. All 
models control for lottery indicators. Impacts expressed in terms of effect sizes. 
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TABLE 5: Impact of Switching to a Private School on Test Score Performance in New 
York for Latinos who Speak English as a Primary and Secondary Language 





English Primary 
(Percentiles) 


(N) 


English Secondary 
(Percentiles) 


(N) 


Year 1 Impact 


-2.56 


[2.54] 


399 


2.30 


[2.91] 


305 


Year 2 Impact 


-1.55 


[3.03] 


342 


3.14 


[3.37] 


290 



* significant at .10 level, 2-tailed test; ** .05 level; *** .01 level. Standard errors reported in parentheses. 
Weighted two-stage least squares regressions performed; treatment status used as instrument. All models control 
for baseline test scores and lottery indicators. In no year are the estimated impacts for the two groups of Latinos 
statistically significantly different from one another. 
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TABLE 6: Impacts of Switching to a Private School on Characteristics of Schools Attended by 

African Americans and Latinos in New York City 





African Americans 
(Percentiles) 


(N) 


Latinos 

(Percentiles) 


(N) 


Diff. In 
Impacts 


School Disruptions 
















Year 1 Impact 


-0.46*** 


[0.13] 


525 


-0.02 


[0.15] 


564 


** 


Year 2 Impact 


-0.28** 


[0.13] 


465 


-0.16 


[0.16] 


523 


— 


Suspensions 
















Year 1 Impact 


0.01 


[0.16] 


515 


-0.13 


[0.11] 


547 


- 


Year 2 Impact 


0.02 


[0.18] 


463 


0.09 


[0.10] 


526 


— 


Dress Rules 
















Year 1 Impact 




[0.10] 


526 


1 3Q*** 


[0.12] 


559 


** 


Year 2 Impact 


1.06*** 


[0.12] 


461 


Q 90*** 


[0.14] 


512 


— 


Hallway Monitors 
















Year 1 Impact 


-0.62*** 


[0.13] 


517 


-0.55*** 


[0.14] 


556 


- 


Year 2 Impact 


-0.67*** 


[0.13] 


464 


-0.10 


[0.15] 


519 


*** 


School Resources 
















Year 1 Impact 


-0.25* 


[0.14] 


530 


-0.15 


[0.14] 


567 


- 


Year 2 Impact 


-0 49*** 


[0.14] 


468 


-0.08 


[0.14] 


526 


** 


School Programs 
















Year 1 Impact 


0.22 


[0.13] 


528 


0.05 


[0.14] 


564 


- 


Year 2 Impact 


-0.16 


[0.15] 


462 


-0.04 


[0.15] 


522 




School Communicate 
w/ Parents 
















Year 1 Impact 


Q y j *** 


[0.13] 


533 


0.30** 


[0.13] 


564 


*** 


Year 2 Impact 


0.78*** 


[0.14] 


469 


0.43*** 


[0.15] 


528 


** 


Amount of Homework 
















Year I Impact 


0.64*** 


. [0.13] 


528 


0 49*** 


[0.03] 


565 


- 


Year 2 Impact 


0.48*** 


[0.13] 


470 


0.33** 


[0.15] 


527 


— 


Class Size 
















Year 1 Impact 


-0.61*** 


[0.13] 


516 


-0.04 


[0.15] 


540 


*** 


Year 2 Impact 


-0.21 


[0.14] 


460 


0.01 


[0.16] 


512 


— 


School Size 
















Year 1 Impact 


-0.88*** 


[0.15] 


367 


-0.47*** 


[0.17] 


364 


** 


Year 2 Impact 


-0 82*** 


[0.15] 


353 


-0.54*** 


[0.15] 


408 


* 


Racial Segregation 
















Year 1 Impact 


-0.03 


[0.13] 


518 


-0.16 


[0.15] 


553 


- 


Year 2 Impact 


-0.13 


[0.15] 


457 


-0.24* 


[0.14] 


513 





* significant at .10 level, 2-tailed test; ** .05 level; *** .01 level. The. last column denotes whether the difference in the 
estimated impacts for African Americans and Latinos is statistically significant. Standard errors reported in parentheses. 
Weighted two-stage least squares regressions performed; treatment status used as instrument. All models control for lottery 
indicators. Impacts expressed in terms of effect sizes. 
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Table 7: Impact of a Voucher Offer on 
African Americans’ Test Scores: Likely Suspects 





Year 1 


Year 2 


(1) 


(2) 


(3) 


(4) 


Treatment 


4.55*** 




3.27** 


3.56** 


School Disruptions 


— 


-0.74 


— 


-5.71*** 


Communication 


— 


-1.86 


— 


-3.33 


School Size 


— 


-2.22 


— 


1,39 


Class Size 


- 


2.06 


-- 


-0.55 


Constant 


-3.38 


-1.41 


0.79 


4.18 


(N) 


624 


624 


497 


497 


Adjusted 


.53 


.52 


.43 


.43 



* significant at .10 level, 2-tailed test; ** .05 level; *** .01 level. Weighted least squares 
regressions performed. Impacts expressed in terms of national percentile rankings. All 
models control for baseline test scores and lottery indicators. Means imputed for missing 
data on co variates drawn from survey. 
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Table 8: Impact of a Voucher Offer on 
Latinos’ Test Scores: Likely Suspects 





Year 1 


Year 2 


(1) 


(2) 


(3) 


(4) 


Treatment 


-0.97 


-1.13 


-0.60 


-0.68 


School Disruptions 


— 


-6.47*** 


— 


-2.44 


Communication 


— 


5.65* 


— 


-1.35 


School Size 


-- 


2.80 


— 


3.25 


Class Size 


- 


6.28* 


- 


14.25*** 


Constant 


20.04 


11.67 


11.47 


3.27 


(N) 


709 


709 


612 


612 


Adjusted 


.44 


.46 


.45 


.47 



* significant at .10 level, 2-tailed test; ** .05 level; *** .01 level. Weighted least squares 
regressions performed. Impacts expressed in terms of national percentile rankings. All 
models control for baseline test scores and lottery indicators. Means imputed for 
missing data on covariates drawn from survey. 
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Table 9: Impact of a Voucher Offer on Test Scores, 
Single Model with Interactions 





Year 1 


Year 2 


(1) 


(2) 


(3) 


(4) 


Treatment 


-0.97 


-1.13 


-0.60 


-0.68 


Treat* AA 


5.53*** 


5.74*** 


3.87* 


4.24* 


African American 


-23.41* 


-13.08 


-10.68 


0.92 


School Disruptions 


— 


-6.47*** 


— 


-2.44 


Communication 


— 


5.65* 


- 


-1.35 


School Size 


— 


2.80 


- 


3.25 


Class Size 


- 


6.28* 


— 


14.25*** 


Disruptions *AA 


— 


5.72** 


— 


-3.27 


Comm.*AA 


— 


-7.51* 


- 


-1.98 


School Size*AA 


— 


-5.02 


— 


-1.86 


Class Size*AA 


-- 


-4.21 


— 


-14.79*** 


Constant 


21.17 


12.04 


12.61 


5.83 


(N) 


1333 


1333 


1109 


1109 


Adjusted 


.48 


.49 


.45 


.46 



* significant at .10 level, 2-tailed test; ** .05 level; *** .01 level. Weighted least squares 
regressions performed. Impacts expressed in terms of national percentile rankings. All 
models control for baseline test scores, lottery indicators, and their associated interactions. 
Only Latinos and African Americans are included in the models. Means imputed for missing 
data on covariates drawn from survey. 
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Table 10: Impact of a Voucher Offer on Test Scores: The Kitchen Sink 





Year 1 


Year 2 


Treatment 


-1.24 


-0.81 


Treat* African American 




4.83** 


African American 


-32.21** 


-1.89 


School Disruptions 


-5.86*** 


-2.20 


Communication 


5.72* 


0.56 


School Size 


2.84 


5.28* 


Class Size 


6.27* 


13 94*** 


Suspensions 


-13 49*** 


-5.52 


Dress Rules 


-2.10 


-1.83 


Hallway Monitors 


-0.93 


-4.96* 


School Resources 


-4.67 


-5.81 


Homework 


6.79** 


7.82** 


Segregation 


1.37 


0.56 


Involve w/ Child 


-8.96*** 


0.06 


Involve w/ School 


-0.78 


-5.92* 


Disruptions *AA 


5.21** 


-3.42 


Comm.*AA 


-10.21** 


-4.16 


School Size*AA 


-6.54* 


-3.49 


Class Size* A A 


-4.68 


-13.67** 


Suspensions*AA 


12.75** 


7.43 


Dress Rules*AA 


3.63 


-0.92 


Hallway* AA 


1.31 


6.85* 


Resources* A A 


14 22*** 


2.18 


Homework*AA 


-7.02* 


-2.21 


Segregation* AA 


-0.97 


-1.16 


Inv. w/ Child*AA 


10.02** 


-4.88 


Inv. w/ School*AA 


1.50 


7.21 


Constant 


22.71 


10.94 


(N) 


1333 


1109 


Adjusted R^ 


.50 


.47 



* significant at .10 level, 2-tailed test; ** .05 level; *** .01 level. Weighted least squares 
regressions performed. Impacts expressed in terms of national percentile rankings. All 
models control for baseline test scores, lottery indicators, and their associated interactions. 
Only Latinos and African Americans are included in the models. Means imputed for missing 
data on covariates drawn from survey. 
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APPENDIX 

Depending upon which year the surveys were administered, indices were constructed from all, or 
a subset, of the items that follow. Response categories are available upon request. 

School Disruptions: “How serious are the following problems at this child’s school? Very 
serious, somewhat serious, or not serious?” Kids destroying property; kids being late for schools; 
kids missing classes; fighting; cheating; racial conflict; racial conflict; guns or other weapons; 
drugs or alcohol. 

Suspensions: “During this past year, was this child ever suspended for disciplinary reasons?” 
Dress Rules: “Are students required to wear a uniform?” “Are certain forms of dress 
forbidden?” 

Hallway Monitors: “Are visitors required to sign in at main office?” “Are hall passes required 
to leave class?” 

School Resources: “At the school this child attends, which of the following programs or 
facilities are available to students?” A computer lab; a library; a gym; a cafeteria; child 
counselors; a nurses’ office. 

School Programs: “At the school this child attends, which of the following programs or 
facilities are available to students?” Special programs for non-English speakers; individual 
tutors; special programs for students with learning problems; special programs for advanced 
learners; a music program; an arts program; an after-school program. 

School Communication with Parents: “Do the following practices exist in this child’s school?” 
Parents informed about student grades halfway through the grading period; parents notified when 
student sent to the office the first time for disruptive behavior; parents speak to classes about 
their jobs; parents participate in instruction; parent open-house or back-to-school night held at 
school; regular parent/teacher conferences held; parents receive notes about this student from this 
child’s teacher’s; parents receive a newsletter about what’s going on in this child’s 
school/classroom. 

Amount of Homework: “Approximately how much homework is assigned on an average day?” 
Class Size: “Approximately how many students are in this child’s class?” 

School Size: “Approximately how large is the school this child attends?” 

Racial Segregation: “What proportion of students in this child’s classroom is minority?” Percent 
responding “everyone” or “90-100 percent.” 

Parental Involvement with Child’s Education: “In the past month, how often did you do the 
following?” Help this child with his or her homework; help this child with reading or math that 
was not part of his or her homework; talk with this child about his or her experiences at school; 
attend school activities; work on school projects. 

Parental Involvement with Child’s School: “How many parent-teacher conferences did you 
attend this school year?” “How many hours have you volunteered in this child’s school this past 
month?” “Are you a member of a PTA or other similar organization (Parent’s Council, for 
example)?” 

Satisfaction Index: “How satisfied are you with the following aspects of this child’s current 
school?” Location of school; school safety; teaching; how much school involves parents; class 
sizes; school facilities; student respect of teachers; how much teachers inform parents of 
students’ progress; how much students can observe religious traditions; parental support for the 
school; discipline; clarity of school goals; teamwork among staff; teaching values; academic 
quality; the sports program; what is taught in the school. 

Overall Grade: “What overall grade would you give this child’s current school?” 
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